Many modern video display devices fall into one of two categories--computer monitor or television monitor. Television monitors are designed for use with broadcast or recorded television signals. The transmission standards for broadcast television signals have been set for decades and include NTSC in the United States and Japan, PAL in much of Europe, and SECAM in France. Analog video tape recording equipment is usually designed to follow one of these standards as well and is sold in each country to conform to the local standard. Other television equipment, such as cameras or camcorders, can be found for use with each of these standards. These transmission standards are well known in the industry and to one skilled in the art. Although each standard differs from the others in terms of number of lines displayed, operating frequencies, and other details, each of these standards shares certain characteristics. For purposes of discussion, the following disclosure will refer to certain specific frequencies and other characteristics of a television standard in terms of NTSC standard values. One skilled in the art will recognize that corresponding values pertain to other television standards and can apply the principles disclosed here to understand the teachings of this invention. Also, transfer of a video signal to a television monitor will be discussed as a transmission, but this transmission might be broadcast or might be transferred through a wired connection.
A single frame of a television video image comprises two fields, each field including a series of scan lines. Referring to FIGS. 1A and 1B, frame 10 includes sequential scan lines 1, 2, 3, 4, . . . 524, 525. The specific number of scan lines, here 525, depends on the specific television standard, here the NTSC standard. Frame 10 is comprised of two fields, odd field 11 and even field 12. Odd field 11 is written to a TV screen as a series of scan lines, odd lines 1, 3, 5, . . . , then even field 12 is written to the TV screen as a series of even scan lines 2, 4, 6, . . . interlaced between the odd scan lines. According to the NTSC standard, each field is written in 1/60th of a second for a total time of 1/30th of a second to display each frame 10. The NTSC standard also specifies how long each scan line may be and some of the electrical characteristics of the signal and of the display. In a conventional television, the signal along each scan line is continuously changing. A television video signal may be recorded on magnetic tape as an analog signal including information about each scan line of each field.
The advent of the digital computer brought about widespread use of computer monitors. Early computers, especially for home use, relied exclusively on television monitors for output, but as computers became more powerful, it became possible to use a much higher-quality display device--a computer monitor. A computer monitor, while in some ways very like a conventional television monitor, has control electronics that are generally much more precise than those found in a conventional television monitor. The computer monitor can position the electron beam much more precisely and so allows for continuous, sequential scanning of the entire frame, line by line, without interlacing. Each line consists of a number of discrete pixels (picture elements). A video image is stored in a computer as a series of digital bytes. Depending on the capabilities of a particular computer or display, this may include only a single bit of information for each pixel (allowing for two colors, typically black or white) or for more powerful systems many bits of information for each pixel. A typical high quality monitor can display 32 bits of information for each pixel.
A number of array patterns for computer displays have become standard in the industry. One particularly common size is 640.times.480 pixels (the VGA standard is this size, and many Macintosh and other computers support VGA and non-VGA monitors with this display resolution). Other common sizes include 320.times.240 (these days generally considered only for low resolution or small image sizes, e.g. one quarter of a 640.times.480 image), 832.times.624, 1024.times.768 and many others. For convenience, the following discussion will refer to 640.times.480 in certain examples but the principles and teachings of this invention pertain to other resolutions as well.
The increasing use of computers has led to the desire to use a standard television to display a computer-generated or -processed image. This is particularly true for a variety of computer game players, which can be connected to a user's television at home or elsewhere. New computers are becoming smaller and more powerful and can be used for games as well as for information processing such as multimedia CDs and communication over the Internet. One example of such a computer is the Pippin, designed and currently being developed by Apple Computer, Inc., of Cupertino, Calif. Certain details of the Pippin architecture are described below by way of example to illustrate the teachings of this invention.
A television monitor suffers from several disadvantages when used to display computer images. Two very significant problems are overscan and flicker.
A typical television transmission is designed to overscan the available display space, so some information is expected to be lost along at least some edges. Where a computer menu or other information is displayed along an edge, typically the top but sometimes the bottom or a left or right edge, loss of even a small portion of the information can make it difficult or impossible to use the computer.
A typical NTSC signal includes 525 scan lines for each frame, although only about 480 of these are shown in a typical display. Some of the remaining lines are part of the vertical blanking interval, which is provided to allow time for the scanning electronics to reset from the extreme low corner of the screen to the opposite, high corner and to allow time to synchronize certain special portions of the television transmission signal. Most of the remaining lines, however, are not displayed because of deliberate overscan.
There are good reasons why a conventional television transmission is designed with overscan. Televisions vary in many ways, including physical scan capabilities, accuracy, curvature, mask or bezel position, manufacturing and component tolerances and component aging. To compensate for this variation, a typical television transmission includes more vertical scan lines of information than can be displayed on a typical television. In addition, a typical transmission includes longer scan lines than can be displayed. Referring to FIG. 2, the resulting visible image 21 displayed is approximately 90% of the transmitted image 22. The result is that for almost all televisions, some information on each edge is thrown away (not displayed) but for almost no televisions is there a blank portion along any edge. However, if a top portion 23 of transmitted image 22 is in the overscan region, it may not be displayed at all.
Flicker can be a significant problem as well. In general, images updated less than 40 times a second on a display have noticeable flicker. Studies have shown that flicker is not noticed by most people if the images are updated faster than 60 times a second (refresh rate of 60-Hz). In interlaced scanning, first all odd lines are scanned from top to bottom, the even lines are skipped. After the vertical retrace, all the even lines skipped in the first scan are scanned from top to bottom. Under NTSC standards, the vertical refresh rate of each field is 60-Hz, resulting in a frame refresh rate of 30-Hz. It is important to note that each scan line is updated at refresh rate of 30-Hz. Under PAL and SECAM standards, the corresponding values are even slower: 50-Hz and 25-Hz.
In addition, images with high vertical contrast tend to flicker noticeably when displayed on interlaced television. An example of this would be a narrow horizontal line, often found in a table of numbers. If the line to be displayed happens to fall just along one scan line on the television, then it will flash on only once every 30th of a second. If that same line was just a bit wider and so fell along an odd scan line and an adjacent even scan line, each line would be displayed alternately so the resulting line would be displayed every 60th of a second (but moving up and down slightly).
Most natural images (scenery, people etc.) do not have a sharp vertical contrast i.e. there is not much difference in intensity between adjacent horizontal lines. This results in an apparent refresh rate of 60-Hz for the TV thereby causing minimal flicker. However, computer generated graphics can have a large intensity vertical contrast (ex. a single pixel black horizontal line on a white background). Since each line is updated at a refresh rate of 30-Hz, the line would flicker noticeably.
Overall, these disadvantages generally are not significant in traditional broadcast television. In a traditional television image, a typical source is some natural scene as captured by a camera. With regard to overscan, if some of the edges are not seen, this is simply perceived as a field-of-view issue, comparable to zooming in or out slightly. With regard to flicker, an object of high vertical contrast, such as a table top, is rarely only one scan line wide. Certain natural objects do contain very fine lines, sometimes found on clothing or perhaps in a fence. Television newscasters avoid clothing with plaids or narrow horizontal stripes because the resulting transmission can result in a striking flicker effect.
Several techniques are used today to address this overscan problem when displaying computer images, but none of these techniques solve all of the problems. Convolution has been used in the past to change the size of an image. Convolution has also been used to convert between interlaced and non-interlaced images. Various convolution schemes used historically are not useful in this instance because they are either too expensive or not sufficiently powerful. Existing solutions include:
1) Recreate content: One simple approach is to recreate content or images that are less than the relevant computer image size, for example 640.times.480 pixels. This however, requires considerable work and many, if not all, computer programs would have to be rewritten. For computer titles that may be used on either a computer or television display, this would require different versions of the program, which inevitably causes problems including sales and inventory management difficulties and consumers being forced to choose one or the other when they might want both.
2) Image reduction by pixel/line dropping: This simple technique drops one out of `n` pixels and/or one out of `n` lines to scale an image. For example, to scale the image by 2/3 just drop 1 out of 3 pixels. Unfortunately, this simplistic algorithm does not yield high quality images. A one-pixel-wide horizontal or vertical line in the source image inadvertently may be omitted entirely in the output image if this technique is used.
3) Bi-linear interpolation: This technique uses weighted average between the two nearest pixels to create an output pixel. This is a cost effective technique resulting in generally good quality images.
4) Multiple tap finite impulse response (FIR) filters: This technique uses multiple tap (up to 65 tap) filters for horizontal and vertical resizing. This results in very good image quality but needs complicated and expensive hardware.
The problem of flicker has been addressed with some limited success by earlier image display schemes. However, none of these schemes can provide the high quality image that consumers now expect when viewing computer images. Existing solutions include:
1) Display only one field: This technique displays only one field of the frame at 60-Hz (the other field is not scanned and is black). However the resulting image is of much lower resolution as every other line is missing (black).
2) Three line convolution: This technique reduces flicker by reducing the contrast between horizontal lines by averaging three adjacent vertical pixels (see FIG. 3). Typically, for interlaced line `p` the new pixel intensity `n` is given by: EQU n=(1/4)*(p-1)+(1/2)*(p)+(1/4)*(p+1) Eqn. 1
where (p-1) is the line above and (p+1) is the line below the line being convolved. This technique is quite effective and reduces flicker to an acceptable level. Note that calculation of the first odd-field scan line is a special case combining only two lines.
3) Two line convolution: This techniques reduces flicker by reducing the contrast between two horizontal lines by averaging two adjacent vertical pixels (see FIG. 4). Typically, for interlaced line `p` the new pixel intensity `n` is given by: EQU n=(1/2)*p+(1/2)*(p+1) Eqn. 2
This technique is not as effective as the three line convolution.